读取图像中文本的能力通常缺乏视觉和语言（V＆L）模型。我们如何学习表现出强烈的场景文本理解（Stu）的V＆L模型？在本文中，我们提出了Prestu，这是一种专门为场景文本理解而设计的简单预训练食谱。Prestu将简单的OCR感知预训练目标与带有现成的OCR信号的大型图像文本数据集结合在一起。我们从经验上证明了这一预训练目标对TextVQA，TextCaps，ST-VQA和Vizwiz-VQA的优越性。我们还研究了哪些因素会影响Stu性能，其中我们强调了在预训练期间图像分辨率和数据集量表的重要性。

我们研究了开发自主代理的问题，这些自主代理可以遵循人类的指示来推断和执行一系列行动以完成基础任务。近年来取得了重大进展，尤其是对于短范围的任务。但是，当涉及具有扩展动作序列的长匹马任务时，代理可以轻松忽略某些指令或陷入长长指令中间，并最终使任务失败。为了应对这一挑战，我们提出了一个基于模型的里程碑的任务跟踪器（M-Track），以指导代理商并监视其进度。具体而言，我们提出了一个里程碑构建器，该建筑商通过导航和交互里程碑标记指令，代理商需要逐步完成，以及一个系统地检查代理商当前里程碑的进度并确定何时继续进行下一个的里程碑检查器。在具有挑战性的Alfred数据集上，我们的M轨道在两个竞争基本模型中，未见成功率的相对成功率显着提高了33％和52％。

Visual question answering (VQA) is challenging not only because the model has to handle multi-modal information, but also because it is just so hard to collect sufficient training examples -- there are too many questions one can ask about an image. As a result, a VQA model trained solely on human-annotated examples could easily over-fit specific question styles or image contents that are being asked, leaving the model largely ignorant about the sheer diversity of questions. Existing methods address this issue primarily by introducing an auxiliary task such as visual grounding, cycle consistency, or debiasing. In this paper, we take a drastically different approach. We found that many of the "unknowns" to the learned VQA model are indeed "known" in the dataset implicitly. For instance, questions asking about the same object in different images are likely paraphrases; the number of detected or annotated objects in an image already provides the answer to the "how many" question, even if the question has not been annotated for that image. Building upon these insights, we present a simple data augmentation pipeline SimpleAug to turn this "known" knowledge into training examples for VQA. We show that these augmented examples can notably improve the learned VQA models' performance, not only on the VQA-CP dataset with language prior shifts but also on the VQA v2 dataset without such shifts. Our method further opens up the door to leverage weakly-labeled or unlabeled images in a principled way to enhance VQA models. Our code and data are publicly available at https://github.com/heendung/simpleAUG.

Recent directions for offensive language detection are hierarchical modeling, identifying the type and the target of offensive language, and interpretability with offensive span annotation and prediction. These improvements are focused on English and do not transfer well to other languages because of cultural and linguistic differences. In this paper, we present the Korean Offensive Language Dataset (KOLD) comprising 40,429 comments, which are annotated hierarchically with the type and the target of offensive language, accompanied by annotations of the corresponding text spans. We collect the comments from NAVER news and YouTube platform and provide the titles of the articles and videos as the context information for the annotation process. We use these annotated comments as training data for Korean BERT and RoBERTa models and find that they are effective at offensiveness detection, target classification, and target span detection while having room for improvement for target group classification and offensive span detection. We discover that the target group distribution differs drastically from the existing English datasets, and observe that providing the context information improves the model performance in offensiveness detection (+0.3), target classification (+1.5), and target group classification (+13.1). We publicly release the dataset and baseline models.

我们介绍韩语了解评估（KLUE）基准。 Klue是8个韩国自然语言理解（nlu）任务的集合，包括主题分类，语言典的相似性，自然语言推断，命名实体识别，关系提取，依赖解析，机器阅读理解和对话状态跟踪。我们从各种源语料库中展开的所有任务，同时尊重版权，以确保任何没有任何限制的人的可访问性。考虑到道德考虑，我们仔细设计了注释协议。随着基准任务和数据，我们为每个任务提供适用的评估指标和微调配方，为每项任务进行预训练语言模型。我们还释放了预用的语言模型（PLM），Klue-Bert和Klue-Roberta，以帮助在KLUE上再现基线模型，从而促进未来的研究。我们通过拟议的Klue基准套件从初步实验中进行了一些有趣的观察，已经证明了这款新的基准套件的有用性。首先，我们找到了klue-roberta-mantring的其他基线，包括多语种plms和现有的开源韩国plms。其次，即使我们从预先预测语料库中取代个人身份信息，我们也会看到性能下降最小，这表明隐私和NLU能力并不彼此可能。最后，我们发现，使用BPE标记与语素级预象的组合，在涉及语素级标记，检测和发电的任务中是有效的。除了加速韩国人NLP研究外，我们的创建Klue的全面文件将有助于将来为其他语言创建类似的资源。 klue在https://klue-benchmark.com上提供。